Overview

simChef is a powerful framework for designing, documenting, and deploying simulation experiments in R.

Goals:

• Provide an intuitive grammar of simulation experiments.
• Reduce the startup time and boilerplate code needed to create simulations.
• Provide a standard and interactive way to document and visualize simulation results.
• Run large simulations on a cluster as easily as small one on a laptop.

Simulation abstractions

Experiment: a collection of related simulation scenarios.

DGP: Data-generating processes, which define the “ground truth” and flexibly generate simulated data.

Method: The main objects of study, along with baselines.

Evaluator: Produce meaningful statistics and summaries of the results.

Visualizer: Output plots, tables, R Markdown, latex, etc. to populate interactive experiment documentation.

Grammar of simulations

experiment <- create_experiment() %>%
  add_dgp(dgp1) %>%
  add_dgp(dgp2) %>%
  add_method(method1) %>%
  add_vary_across(
    dgp = dgp1,
    n = c(100, 1000, 10000)
  ) %>%
  add_vary_across(
    method = method1,
    lambda = c(0.1, 0.5, 1.0)
  )

results <- experiment %>%
  run_experiment()

Interactive documentation

Roadmap

• Nested parallelism.
• A sibling package of flexible and modular DGPs.
• Ongoing usability improvements.
• Please submit issues and feature requests on GitHub.

Acknowledgements

We acknowledge a number of funding sources and grants that help support our research, including the Weill Neurohub, C3.ai, and the NSF.